Experimental Results on Minimizing Communication

In order to minimize total execution time of parallel algorithms for coarse-grained parallel computers or networks of workstations, the costs of local computation and communication costs must be balanced. In some cases minimizing the total number of global communication operations may not be the best way to achieve this. There have been several recent papers dealing with parallel algorithms of small communication cost under diierent models. In particular , we study an approach based on minimizing the number of communication phases. In this approach, we design algorithms of multi-threads with a small number of communication rounds between the threads of computation. Several authors have argued for this approach from theoretical perspectives. In this work, we study the question empirically by comparing the performance, on a collection of networked workstations, of two similar algorithms for solving the shortest path problem. Although the algorithms have similar overall communication and local computation requirements, the number of communication phases is much more sharply bound in one of them. Our results strongly support theoretical predictions that communication phases incur large setup costs and should therefore be minimized.